MLE-Guided Parameter Search for Task Loss Minimization in Neural Sequence Modeling

Neural autoregressive sequence models are used to generate sequences in a variety of natural language processing (NLP) tasks, where they evaluated according sequence-level task losses. These typically trained with maximum likelihood estimation, which ignores the loss, yet empirically performs well as surrogate objective. Typical approaches directly optimizing loss such policy gradient and minimum risk training based around sampling space obtain candidate update directions that scored on single sequence. In this paper, we develop an alternative method random search parameter leverages access gradient. We propose guided (MGS), samples from distribution over is mixture current parameters gradient, each direction weighted by its improvement loss. MGS shifts space, scores candidates using losses pooled multiple sequences. Our experiments show capable losses, substantial reductions repetition non-termination completion, similar improvements those machine translation.